Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Batch Queries. Support multiple lifts. #100

Merged
merged 1 commit into from
May 16, 2022

Conversation

deusaquilus
Copy link
Contributor

@deusaquilus deusaquilus commented May 16, 2022

Resolves #101

The current system that does batch queries makes two big assumptions for any given batch query:

liftQuery[T](...).foreach(t => query[R](.filter(...)).update/insert/delete(Value)(...)
  1. That type T and R are the same (in the BatchQueryExecution this parameter is I).
  2. That there are no other lifts in the entire batch query other the the ones given by the initial liftQuery construct.

Quite a few refactorings in BatchQueryExecution are needed in order to remove these assumptions mostly centering around BatchQueryExecution but bits in StaticTranslationMacro and dynamic parts of QueryExecution as well.

  1. In order to know how to lift the T of liftQuery, you need to know what InjectableEagerPlanters (as well as the CaseClass/Scalar Ast value of corresponding to T) to produce from it. Currently this is being as part of the BatchQueryExecution process with the assumption that it is the same as the I variable which is the output of the batch-query (meaning it is the same as the query[R] part i.e. I is R. However, as we have seen before this cannot be the case. Now, if we know the type in liftQuery[T] in the BatchQueryExecution process, we can use the T in there instead, however what do you do if the query is dynamic and T is no longer available due to type-erasure.
    Instead, we need to summon the InjectableEagerExprs (and CaseClass/ScalarAst) when the liftQuery of the batch-query is called and store them inside the EagerEntitiesPlanter. Hence the fieldGetters and the fieldClass types are added to EagerEntitiesPlanter and are then used in BatchQueryExecution to extract these properites.
  2. The case where the original liftQuery only holds a scalar (e.g. liftQuery(List(1,2,3))) needs to be accounted for as well. Since the type information is not as important (since the encoder instance is already available on the resulting EagerListPlanter) we can reconstruct the needed functionality just by synthesizing a fake InjectableEagerPlanter that "injects" each scalar value of the list.
  3. Since we need to allow other lifts besides the InjectableEagerPlanters created by the EagerEntitiesPlanter/EagerListPlanter of the liftQuery (we call this the PrimaryLift), we need a pipeline for "Secondary Lifts" to be passed into the methods used for static and dynamic resolution of the query i.e. StaticState and PrepareDynamicExecution. Also, because one of these additional lifts can be another liftQuery clause...

    i.e. one that converts to a set operation e.g.

    liftQuery(...).foreach(p => query[P].filter( p => liftQuery(List(1,2,3).contains(p.id)))
    this effectively means that we need to "particularize" the query in the BatchQueryExecution as well because the secondary liftQuery clause need another placeholder ? for every member that it has (same with all IN SET liftQuery situations).
  4. Also, I noticed that when I added the InjectableEagerPlanters to the EagerEntitiesPlanter the ExprAccumulate in the findUnquotes would recurse into the EagerEntitiesPlanter and find the InjectableEagerPlanters inside hence pulling them out. This is not the desired behavior since these should only be extracted inside of the BatchQueryExecution process. Therefore I modified the ExprAccumulate process to pass an argument whether to recurse or not when a the type of expression that is searched for (e.g. case expr @ PlanterExpr.Uprootable(planter) => planter in findUnquotes) is found.

@deusaquilus deusaquilus force-pushed the dynamic-batch-enhance branch from 53e8228 to 0d43431 Compare May 16, 2022 03:38
@deusaquilus deusaquilus merged commit fb14ec1 into master May 16, 2022
@deusaquilus deusaquilus deleted the dynamic-batch-enhance branch May 16, 2022 04:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Batch-Query Use-Cases not supported
1 participant